Shrink and Eliminate: A Study of Post-Training Quantization and Repeated Operations Elimination in RNN Models

Recurrent neural networks (RNNs) are (NN) designed for time-series applications. There is a growing interest in running RNNs to support these applications on edge devices. However, have large memory and computational demands that make them challenging implement Quantization used shrink the size needs of such models by decreasing weights activation precision. Further, delta method increases sparsity vectors relying temporal relationship between successive input sequences eliminate repeated computations accesses. In this paper, we study effect quantization LSTM-, GRU-, LiGRU-, SRU-based RNN speech recognition TIMIT dataset. We show how apply post-training with minimal increase error skipping selected paths. addition, integer precision leads considerable if applied. Then, propose increasing while minimizing maximizing percentage eliminated computations. The proposed managed compress four more than 85%, an 0.6, 0, 2.1, 0.2 points, respectively. By applying quantized models, 50% operations can be eliminated, most cases only minor error. Comparing each other under method, found compressed LSTM-based most-optimum solutions at low-error-rates constraints. smallest size, suitable when higher rates acceptable, LiGRU-based highest number operations.